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ABSTRACT 


Fluctuation  theory  is  concerned  with  the  study  of  extreme  values 
of  sums  of  independent,  arbitrary-valued  random  variables.  Simple 
but  powerful  combinatorial  methods  due  chiefly  to  E.  S.  Andersen, 
F.  Spitzer,  and  W.  Feller  have  recently  provided  an  easy  method  of 
attack  on  these  problems.  However,  operations  research  models  are 
concerned  with  fluctuations  of  various  economic  returns  which  are 
earned  at  random  points  in  time,  and  whose  increments  are  corre¬ 
lated  with  the  interval  since  the  last  payoff.  Our  generalization 
considers  the  fluctuations  of  a  cumulative  reward  process,  defined 
on  an  underlying  renewal  process.  Most  of  the  classical  results 
carry  through,  including  Weiner-Hopf  type  factorization,  an 
Andersen-Pollaczek-Spitzer  type  identity,  and  certain  Waldian- 
Pollaczek  results.  As  applications,  we  find  the  distribution  of 
the  maximum  return  over  a  mixed  index-epoch  horizon,  and  show  how 
certain  general  results  for  the  GI/G/1  queue  follow  directly  from 
the  various  three-dimensional  ladder  distributions. 


FLUCTUATIONS  OF  A  RENEWAL- REWARD  PROCESS* 

by 

William  S.  Jewell 


1.  INTRODUCTION 

Fluctuation  theory  is  concerned  with  the  extreme  values  of  the  sums 

n 

(1)  p  -  0  ;  p  «  l  £  (n  «  1,2,  ...) 

i*l 

where  the  £  are  independent  random  variables  with  common  distribution  Q  .  In 
the  case  where  the  £  are  nonnegative,  we  have  a  simple  problem  in  renewal  theory. 
However,  when  arbitrary  values  of  the  £^  are  allowed,  the  theory  becomes  more 
difficult;  it  is  only  recently  that  simple,  but  powerful,  combinatorial  methods, 
developed  by  E.  S.  Andersen  [1,2,3],  and  extended  and  simplified  by  F.  Spitzer 
[17,18,19,20],  and  W.  Feller  [9],  have  been  discovered  for  the  general  case.  Read¬ 
able  summaries  of  this  modern  approach  and  more  detailed  references  may  be  found  in 
Chapters  12  and  18  of  [8],  [14],  and  Chapter  6  of  [16], 

An  interesting  generalization  results  if  one  thinks  of  the  p^  in  (1)  as  a 
cumulative ,  or  reward  process,  "earned"  at  epochs  in  discrete  time,  n  «  1,2, 
we  then  allow  these  reward  epochs  to  take  on  a  random  nature  by  associating  them 
with  the  epochs  of  a  renewal  process.  Thus,  our  generalization  is  the  study  of  the 
fluctuations  of  a  reward  process  imbedded  in  a  renewal  process. 

Let  the  underlying  renewal  process  consist  of  nonnegative  intervals  {x^}  > 

(i  =  1,2,  ...)  ,  with  common  distribution  A  ;  then,  the  location  of  the  n*"*1  epoch 
is  at 


t  bi 
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(2) 


n 


(n  *  1,2,  . . .) 


and  the  total  reward  accumulated  to  that  point  is  defined  as: 


(3) 


n 


n 


l 

i«l 


MV 


At  times  between  these  epochs,  the  total  reward  remains  constant  at  its  last  previous 
value,  i.e.,  we  define: 

/  0  n(t)  =  0 


(4) 


P(t) 


n(t) 


n(t) 

l  n(t)  >  0 


■i-l 


(t  >  0) 


with 

(5)  n(t)  =  sup  (k  |  »k  _<  t) 

as  the  number  of  epochs  in  [ 0 , t ]  .  A  typical  realization  of  p(t)  is  shown  in 
Figure  1. 

Intuitively  we  may  think  of  as  an  interval  reward,  possibly  dependent 

on  ,  whicn  is  "earned"  at  the  end  of  interval  i  ;  p(t)  is  then  the  total  reward 

earned  in  [0,  t  ]  .  For  this  generalization,  we  must  specify  the  joint  distribution, 

/  i  =  1,2,  .. 

(6)  Q(t;y)  =  p{t1  1  t  ;  ^  £  y)  l  t  >^0 

'—oo  <  y  <  —oo 

whose  marginal  distributions  are  A(t)  and  Q(y)  .  (We  assume  A(0)  <  1  , 

=  0}  <  1  ,  and  Q(t;y)  is  an  honest  distribution.) 

Since  the  imbedded  reward  process  {p  =  p(j  )}  is  the  same  as  the  fluctuation 

n  n 

process  of  (1),  i.e interval  rewards  are  independent  from  epoch  to  epoch,  much  of 


FIGURE  1.  TYPICAL  REALIZATION  OF  A  (CUMULATIVE)  REWARD  PROCESS, 
DEFINED  OVER  AN  UNDERLYING  RENEWAL  PROCESS. 
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the  simplicity  of  the  original  model  is  retained.  However,  the  addition  of  the 
rerewal  process  {f^}  as  a  "random  clock"  leads  naturally,  as  we  shall  see,  to  more 
complex  fluctuation  models,  particularly  in  queueing  theory. 

The  process  p(t)  could  also  be  thought  of  as  a  two-dimensional  random  walk 
(p^,  ir)  in  which  the  second  component  can  only  increase.  However,  this  generality 
will  not  be  needed  in  what  follows. 


2.  THE  IMBEDDED  RENEWAL  PROCESS:  LADDER  POINTS 

The  key  to  analysis  of  fluctuation  processes  lies  in  a  concept  first  used  by 
D.  Blackwell  [7]:  the  ladder  point.  A  (strict  ascending)  ladder  point  is  said  to 
occur  at  index  n  and  epoch  $  iff 


(7) 


p(in)  >  p(t)  (0  <  t  <  *n)  ; 


the  ladder  process  obtained  by  connecting  the  ladder  points  in  the  obvious  way  is 
then  an  upper  envelope ,  or  maximum ,  to  the  process  p(t)  ,  (see  Figure  2).  The 
origin  is  sometimes  considered  as  a  zero^  ladder  point. 

We  distinguish  three  ladder  variables  in  Figure  2:  the  index  at  which  the 
record  value  took  place;  the  epoch  associated  with  that  index;  and  the  height  of 
that  record  value.  For  the  first  (strict  ascending)  ladder  point,  we  have: 


(i) 

an  index 

(8) 

(ii) 

an  epoch 

and 

(iii) 

a  height 

min  {n  I  p(s  )  >  0}  >  0  , 
n 

•vj  -  0  • 

p(+l)‘  Pv1  ”  0  • 


with  joint  distribution 


F(n,t;y)  -  P{v1  « 


/n  *  1,2,  ...\ 

£  t  ;  A  <_  y}  .  I  0  <_  t  <  ®  I 

1  \0  <  y  <  -  / 


(9) 


n 


i 


5 


p(t) 


Ladder  Point  2 


T 


FIGURE  2.  REWARD  PROCESS  SHOWING  (STRICT  ASCENDING)  LADDER  POINTS  AND  LADDER 
PROCESS,  (STRICT  ASCENDING)  LADDER  INDICES,  EPOCHS,  AND  HEIGHTS. 


6 


It  follows  from  the  definitions  and  Figure  2  that  the  distributions  of  all  the 
ladder  variables  could  be  calculated  from  (9) ,  since  the  ladder  points  constitute 
ccn  imbedded  (three-dimensional)  renewal  process ,  in  which  each  new  record  value 
occurs  "from  scratch,"  starting  at  the  previous  value;  i.e.,  the  triplet 
is  independent  and  identically  distributed  for  all  i  =  1,2,  ...  An  obvious,  but 
important,  identity  relates  the  transform  of  the  distribution  of  the  r1"*1  ladder 
point  to  that  of  the  first  ladder  point: 


(10) 


{(Vl+V2+* * *+Vr)  -s  (4>^+(J>2+. .  .+^r)  -p(A^+A2+. .  .+Ar>  | 
s  -e  -e  >  = 

.  U,\‘cVpAij]r  . 


(r  =  1,2,  ...) 


We  shall  henceforth  drop  the  index  when  discussing  an  arbitrary  triplet  (v,<|>;A)  . 

In  the  most  interesting  cases,  the  ladder  process  is  defective ,  i.e.,  at  some 

index,  further  record  values  cease  to  occur;  the  probability  of  this  occuring  is  the 

00 

defect ,  1  -  l  F(n,<=°;°°)  . 

n=l 

By  weakening,  reversing,  or  weakening  and  reversing  the  inequality  in  (7), 
four  different  ladder  processes  can  be  obtained.  To  distinguish  between  them,  we 
shall  use  four  different  superscripts  on  the  variables  and  on  the  corresponding 
distribution  F  : 


(i) 

strict  ascending 

(>)  ,  F  +, 

a+  > 

0) 

(ii) 

weak  ascending 

(>)  ,  F®, 

(A®! 

0) 

(iii) 

strict  descending 

(<)  ,  F  ", 

U"  < 

0) 

(iv) 

weak  descending 

(1)  ,  F®, 

(A®  £ 

0) 

Note  the  different  regions  of  definition  for  the  ladder  height. 

We  shall  see  that  the  processes  (i)  and  (iv),  or  (ii)  and  (iii)  are,  in  a 
certain  sense,  duals  of  each  other;  of  course  the  distinction  between  (i)  and  (ii) , 


'  *»i 
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or  (iii)  and  (iv)  vanishes  when  Q  is  continuous.  A  defective  weak  process  implies 
a  defective  strong  one,  and  vice  versa;  however,  not  both  the  ascending  and  the 
descending  processes  can  be  defective. 


3.  DUALITY:  THE  REFLECTION  PRINCIPLE 

Determination  of  explicit  forms  for  the  ladder  distributions  depends  on  two 
fundamental  principles.  The  first  of  these,  whose  proof  is  trivial,  is: 

The  Reflection  Principle 

Consider  any  realization  ('t1»^2.*T2*^2**“*Tn’^n^  °f  a 
reward  process,  for  some  fixed  n  ,  and  define  an  "image" 

(12)  process  by  taking  the  variables  in  the  reverse  sequence 

(Tn*?n*Tn-l’^n-l*  **  *»Ti»^i^  *  Anh  event  defined, by  reali¬ 
zations  of  the  original  process  has  the  same  probability  as 

the  corresponding  event  defined  by  the  image  process. 

The  term  reflection  principle  arises  from  the  fact  that  the  image  process  is  the  one 
seen  by  someone  who  "stands"  on  the  point  (p^.j^)  °f  Figure  1,  and  reverses  the 
orientation  of  both  axes.  (The  rewards  also  appear  as  if  earned  at  the  beginning 
of  the  interval,  but  this  is  unimportant.) 

Figure  3  shows  an  important  application  of  the  reflection  principle  to  prove: 


(13) 


Hpn  >  °n-l  •  °n  >  pn-2 . °n  ’  “l  :  0  ‘  \  ±  y  :  >n  -  t) 

■  P{p^  '  0  ,  p2  >  0,  ,  pn_^  >0;0<pn<_y;  <_  t} 


( 


n  ■  1,2 

0  <  y  < 
0  <  t  < 


The  left  hand  side  of  (13)  is  the  probability  that  n  is  the  index  of  some  ladder 
point  whose  epoch  is  less  than  or  equal  to  t  ,  and  whose  height  is  less  than  or 
equal  to  y  .  This  is  the  three-dimensional  analogue  of  the  renewal  function,  M  , 
(mean  counting  function)  of  classical  renewal  theory,  so  we  set 

(14)  M+(n,t;y)  -  LHS  of  (13) 
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Define  three-fold  transforms: 


m+(s,s;p) 


n 


n=l 


00  00 

/■-  f: 


-py  a2 


t.y 


M+(n,t;y) 


(15)  f+(s,s;p)  =  I  3n 

n=l 


then  from  (10),  and  the  fact  that  n  must  be  the  first  or  the  second  or  the  .... 
ladder  point: 


/•-  / 


e-Py  d2 
t  1 


F+(n,t;y) 


=  E{zVe-S*e~PX. 


(16) 


m 


f+  + 


(f+)2  + 


(fV  + 


I 


in  the  appropriate  region  of  the  complex  space  (2,s,p)  ,  analogous  to  the  usual 
one-dimensional  result  in  renewal  theory. 

th 

The  right  hand  side  of  (13),  on  the  other  hand,  is  a  measure  for  the  n  epoch 
(not  necessarily  a  ladder  epoch) ,  with  »n  <_  t  and  pR  <_  y  ,  for  a  process  which 
has  yet  to  reach  its  first  weak  descending  ladder  point  (excluding  the  origin)  — 
i.e.,  has  yet  to  go  nonpositive  (excluding  [0,i^])  .  Thus: 

P{v®-l;$®<t}  -  Q(t;0+} 


(17) 


P{v®-n+l;$  ®<t] 


t  ® 

//. 

U“0  x*o 


Q(t-u;x)d 


u,x 


M+ (n,u;x) 


(n  -  2,3,  ...) 


Similar  duality  relationships  exist  between 


and 


etc. 


4.  DECOMPOSITION  OF  DUAL  PROCESSES 

With  this  dual  interpretation  of  M+  ,  its  complete  form,  and  that  of  the  dual 
first-passage  (weak  descending  ladder)  distribution  F®  ,  can  be  expressed  in 
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recursion  relations  in  terms  of  the  joint  distribution  Q  .  From  the  definitions: 


(18) 


(19)  M  (n+l,t;y) 


M  (l,t;y)  -  Q(t;y)  -  Q(t;0) 


t  °° 


0  <  t  <  00 

0  <  y  <  oo 

l n  =  1,2,3, 


If. 

u=o  x=o 


[Q(t-u;y-x)  - 


Q(t-u ; -x)  ] dj"  M+(n,u;x) 

U  |  A 


which  can  be  built  up  recursively.  For  the  nonpositive  values  of  y  ,  we  have 
simply 


(20) 


(21) 


Q(l,t; 


y)  =  Q(t;y) 


F°(n+l,t;y) 


t  00 

//. 

u=o  x=o 


2  + 

Q(t-u;y-x)  d  M  (n,u;x) 

U  |  A 


0  <  t  <  00 
-00  <  y  <  0 

n  =  1,2,3,  . .  J 


+  © 

Because  of  the  nonoverlapping  ranges  of  M  and  F  ,  except  for  the  continuation 


(22) 


F®(n,t;y)  =  F°(n,t;0)  ,  (y> 


0) 


there  is  no  ambiguity  if  (18)  -  (21)  are  written  together  as: 


(23) 


M+(l,t;y)  +  F^(l,t;y)  *  Q(t;y) 


t  _  ” 


(24)  M+(n+l,t;y)  +  F®(n+l,t;y)  *  J  J  Q(t-u;y-x)  d^  xM+(n,u;x) 


0  <  t  <  °° 

—00  <  y  <  +00 

n  1,2,  ... 


u=o  x=o 


Setting  t=°°  gives  the  usual  representation  of  an  ordinary  fluctuation  process  [8] 
Taking  the  transform  of  (23)  -  (24),  using  definitions  similar  to  (15)  and 


(25) 


00  oo 

q (s;p)  -  J e  8t  j " e  Py  d^yQ(t;y)  =  E{e  ST  P^}  , 
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we  obtain  the  neater  result: 

(26)  m+(s,s;p)  +  £&(zts;p)  -  3q(s;p) [l-hn+(z,8;p) ] 

or 

[l-hn+(3, s;p)  ]  [l-3q(8jp)]  -  1-f  ®(s, a;p) 

from  which  the  special  cases  can  be  easily  deduced. 

By  setting  z  ■  [l/q(s;p)  ]  in  (26)  or  (28)  (which  operation  can  be  justified, 
see  [8]),  we  obtain  a  generalization  of  Wald's  Identity : 

<27)  sjteJ"*8'"*}'1 

which  clearly  holds  for  all  types  of  ladder  points  (11). 

5.  WIENER-HOPF  FACTORIZATION 

If  (16)  is  substituted  in  (26)  we  obtain  the  striking  result 
(28)  [l-f+(z, s;p)  ]  [1-f  ^3,s;p)]  -  l-2q(s;p)  , 

that  is, 

[l-2Ete-8T-P5)]  -  [l-E{2uVa*+-PX+}j  [l-E^  ^  Q-PX+|1  . 

This  decomposition  of  the  honest  distribution  Q  into  two  distributions,  F+  and 
F  ®  ,  one  of  which  may  be  defective,  is  a  Wiener-Hopf  factorization;  in  many  simple 
cases  of  Interest,  such  as  rational  transforms,  the  factorization  can  be  performed 
explicitly,  using  knowledge  of  the  regions  of  analyticity  in  the  complex  p-plane 
which  follow  from  the  different  domains  of  definition  of  F+  and  F®  .  (See  [8], 
[15],  and  [16]  for  examples  of  factorization.) 

The  asymmetry  in  (28)  can  be  removed  by  defining  a  distribution  of  first 
return  to  the  origin  as  the  limit  of  y  t  O’  of  F®(n,t;y)  ,  viz: 
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(29) 


F®(n, t)  -  P{\^-n,4>®<t}  -  P{p1>0,p2>0 . pn-l>0;pn=°*  ‘n-^ 

=  p{p1<o,p2<o,  ....  pn_1<°;pn“°* • 


Taking  transforms  and  using  the  obvious: 


(30) 


'1+m  ®)  =  (1-hP)  (1+m  )  , 


the  symmetric  form  of  (28)  is  obtained 


(31) 


[l-f+]  [1-f']  [l-f°]  =  1  -  zq  ; 


this  distinction  vanishes  when  Q(y)  is  continuous. 

6.  THE  CYCLIC  PERMUTATION  PRINCIPLE:  THE  EXPLICIT  FACTORIZATION  IDENTITY 

The  second  basic  combinatorial  result  is  based  upon  the  n  possible  cyclic 
permutations  of  the  reward  structure. 

Define  the  event: 


(32) 


£r(n,t;y) 


{n  is  the  r1"*1  ladder  index;  j  <t;p  <y} 

n—  n—' 


n  =  1,2,  . . .' 
t  >  9 
y  >  0 


Then: 

Cyclic  Permutation  Principle 


(33) 


w 

“  p{*n  It  ;  0  <  Pniy)  B  \  ^  P{<?  (n,t;y)} 

r»l 


Proof: 

The  proof  of  (33)  follows  exactly  a  proof  of  Feller  for  the  case  t~°°  ,  y-°° 
([8],  p.  395).  Q.E.D. 

The  first  probability  in  (33)  is  governed  by  the  n-fold  convolution  of  Q  , 
and  the  probability  on  the  RHS  is  the  r-fold  convolution  of  F+  .  Hence,  taking 
transforms  over  the  appropriate  range: 
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OO  00 


l  f-  f  e"St  f  e"py  Qn#(t;y)  =  S  7  [f+(2,s;p)]r  = 

n=l  n  J  J+  C,y  r-1 


■  -fcn[l-f  (s,s;p) ]  , 


we  obtain  an  explicit  factorization  of  (28): 


00  00 


(34) 


f+(a,s;p) 


i  t  /•  '•  /. 


e'Py 


=  1-e 


Various  factorizations  of  this  type  were  first  obtained  by  Andersen,  Pollaczek, 
and  Spitzer,  so  it  is  perhaps  appropriate  to  refer  to  (34)  as  an  Andersen-Pollaczek- 
Spitzer  Factorization  Identity.  (See  discussion  in  [21]  and  [6].)- 

By  analogy  with  previous  results,  (34)  becomes  an  omnibus  formula  for  all 
first-passage  distributions  according  to  the  following  schemer 


(35) 


showing  the  appropriate  range  of  y-integration 
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In  fact, 


d-f+]  [i-f©] 


r  +■ 

oo  oo  o 

-i  £  /••“  /.  *  / 

0  0 


e~Py  dv  tQn*^t,y^ 

y  t L 


-I  “  [q(s,p)]n 


=  l-aq (s ,p) 


which  provides  independent  verification  of  (28).  Or,  conversely,  taking  logarithms 
of  (26),  expanding  £n[l-sq(s,p) ]  in  powers  of  z  ,  and  factoring  in  the  complex 
p- plane  will  provide  independent  verification  of  the  identity  (34) . 

It  should  be  noted  that  integration  with  respect  to  t  does  not  require 
factorization  in  the  above  sense. 


7.  CLASSIFICATION  OF  BEHAVIOR 


Because  the  reward  process  defined  on  the  indices  of  the  epochs  is  just  the 
classic  random  walk  in  ,  the  basic  classification  scheme  for  these  fluctuation 
processes  still  must  hold,  with  some  slight  modification  in  the  case  where  the 
interval  between  epochs  has  infinite  mean.  We  summarize  these  known  results  below, 
for  completeness. 


Assume  for  the  moment  that  F^  is  an  honest  distribution;  from  (34),  or  (20)  - 


(21): 


E(*©1  -  , 
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with 


£  ■  l  F+(j,»;oo)<i  . 

j=l 


Similar  relationships  hold  between  any  dual  pairs  of  ladder  processes.  Note  that 
(38)  is  meaningless  unless  E{£}  £  0  ,  and  that  all  means  except  possibly  (38)  are 
infinite  if  F+  is  honest. 

On  the  other  hand,  from  (34)  -  (35) 


(39) 


with 


(40) 


S+  -  l  £  [1-Qn#(“;0)]  =  l  ^P{pn>0}  , 


n=l 


n-1 


and  similarly  for  F®  ,  which  implies 


(41) 


©  = 


=  l  -  P{p  <0} 
L  n  n— 


n=l 


must  diverge  if  F®  is  honest. 

Note  that  mean  ladder  epochs  and  heights  may  be  infinite,  in  the  honest  cases, 
if  E{t}  or  E{£}  is  infinite. 

Additionally,  if  both  ladder  distributions  are  honest,  it  follows  that  E{£}  =  0 
(and  vice  versa),  and  then  all  ladder  epochs  and  indices  are  infinite.  For  the  mean 
ladder  heights,  however,  differentiating  (28)  gives  the  well-known  result: 


(42) 


E{X+)  •  E{-A^}  =  +HVU)  , 


so  that  the  mean  ladder  heights  are  both  finite,  if  the  variance  of  £  is  finite. 


In  fact,  using  the  Central  Limit  Theorem  and  a  Tauberian  theorem  on  (34),  one  can 
show  the  following  explicit  result,  due  to  Spitzer  (see  [8]): 


(43) 


E{  A  } 


» 


where 


is  the  conditionally  convergent  series 


00  00 

(44)  C+  =  l  -  [P{p  >0}-h]  =  -c®  =  -I  -  [p{p  <0  }-is] 

u ,  n  n  ,  n  n— 

n=l  n=l 

Summarizing  these  results  for  the  dual  pair  {F+,F®}  ,  we  have  the  classifi¬ 
cation  scheme  shown  in  Figure  4. 

8.  FURTHER  IDENTITIES 

If  a  ladder  distribution  is  honest,  and  the  required  moments  are  finite,  then 
continued  manipulation  of  (26)  will  produce  the  following  "Waldian"  identities  for 


the  moments 

of  the 

record 

values : 

(45) 

E{  v }  = 

1+E{ total  number  of  ladder  points 

sdual 

in  the  dual,  ladder  process}  =  e 

(46) 

E{  <j> }  -  E{  t  }  • 

E  { v } 

(47) 

E { A }  =  E{ £ }  • 

E{  v} 

(48) 

v{$} 

=  2E{t } 

•  E{vA}  +  E{v}  • 

V{t  }  -  [E(t)]2E{v2} 

(49) 

v{  a} 

=  2E{ £} 

•  E { v cp }  +  E{v}  • 

V  {  £ }  -  [E(£)]2E{v2} 

E{  <f>A }  =  E{£}  •  E{vA}  +  E{  t  }  •  E{  +  E{v}  •  EUt}  - 
E{ v }  •  E { t }  •  E{ £ }  -  E{ t }  •  E{  £}  •  E{v2}  , 


(50) 


etc.  (46)  and  (47)  may  be  contrasted  with  the  expressions  for  the  means  of  the 
ordinary  values  (see,  for  example,  111]) 


(51) 


E{ t  }  =  E { t }  •  n  , 
n 


and 


(52) 


E{p  }  =  E{ £ }  •  n 
n 


or 


(53) 


lim 

t"H» 


E{p(t)  }  E{g} 
t  “  E{ t } 


9 .  EXTREME  VALUES 

As  an  example  of  other  random  variables  in  a  fluctuation  process  which  are  of 
interest,  consider  the  extreme  values  of  reward  attained  during  some  fixed  horizon. 
For  instance,  the  maximum  over  the  mixed  horizon  (N;T)  ,  eN<T  ,  is  defined  as: 

<54)  4;i-0”“Nlpk 1  -ki”  •  (o  '") 

From  Figure  2,  we  see  that  this  extremum  process  is  just  the  upper  envelope  created 
by  the  strict  ascending  ladder  points,  including  the  origin,  which  occur  in  [0,N] 
and  [0,T]  . 

Defining 


(55) 


<  »  the  first  index  at  which  the  maximum  in  (54)  is  attained 


we  have 


(56)  *  =  the  first  epoch  at  which  the  maximum  in  (54)  is  attained 


and  the  identity  eN>T  =  p^  . 


From  first  principles: 


(57) 


E+(k,t;y|N;T)  =  P{K=k,  f^<t;  P<  <y  |  N  ;T} 
[l-F+(N-k;T-t;»)][6kol(t)l(y)+M+(k,t;y)] 


0  ^k  <  N 
0  <  t  <  T 
0  <  y  <  °» 
N  -  0,1,2, 
lO  <  T  <  ™ 


with  6qo  =  1  ,  6ko  =  0  (k^o)  ,  and  l(t)  as  the  unit  step. 
Using  (10)  and  (34),  the  five-fold  transform  of  (57)  is: 


e  ( z,s;p  w;q) 


(58) 


r  N  f  -qT  i  k  -S'<  -Pp<) 
l  U  I  e  H  E <z  e  e  >dT 

N=o  J  (  I 

o  ' 

OO  CO 

00  n  /*  /* 

v  w  /  -at  /  n  -st  -p y.,2  „n*.  . 

-I  —  I  e  H  /  [1-2  e  e  ^]dt  Q  (t;y) 

n=i  j  j  + 


y*o 

<?[1  -  w] 


from  which  many  previously  known  special  cases  can  be  derived.  (See,  for  example, 
Chapter  6,  [16]  or  Chapter  18,  [8].) 

In  particular,  we  mention  the  generalization  of  well-known  formulas  for  the 
mean  index,  epoch,  and  value  of  the  maximum  attained  in  (N;T)  : 


(59) 


N  T  *  » 

E{k  |  N;T}  -  l  I  I  d2  Qn#(t; 

n*l  ^  J  +  ,y 


y) 


t*o  y-o 


(60) 


E{  i  |  N;T} 

K  ' 


NT  oo 


y) 


t«o  y-o 


N  T  ^  » 


(61) 


EtP*  '  N;T>  'Jin  J  /  +  y  dc,yqn'<t:y)  • 


t“0  y-o 
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Because  of  its  intimate  relationship  with  the  strict  ascending  ladder  process* 
the  maximum  process  also  drifts  off  to  +  00  ,  as  N  and  T  increase  without  limit, 
unless  E{£)  <  0  .  In  this  case  <  is  finite,  with  probability  one,  and  we  have 
simply: 


03  00 


-8  S  -Dp 

(62)  E{zKe  *  K 


-U  j  f  +  Q"*(tiy> 


t=o  y=o 


(N"°°,T"<»;E{^}<0)  .  E{k|00,00}  is  finite,  but  the  magnitudes  of  Efi^)00,00}  and 

E{pkI“»°°}  depend  upon  further  assumption  about  Q  ,  By  comparison  with  (34)  and 
(39),  we  also  see  that  in  the  infinite-horizon  case: 


(63)  E+(k,t;yj“>;°°)  -  [1-F+] [6kQl(t) l(y)+M+(k, t;y) ]  , 


which  could  have  been  predicted  from  first  principles. 

Another  interesting  case  occurs  if  Q(t,y)  is  symmetric*  about  y=o  for  all 
values  of  t  .  Then,  by  direct  summation  of  (62),  the  index  and  epoch  of  the 
maximum  have  the  transform: 


(64) 


a+(2,s;o|w;q)  - 


q ( 1-w)  Jl-wz&(8+q) 


—q  T 

where  a(e)  -  E{e  }  .  No  finite  maximum  exists  for  this  oscillating  case  as  N 
and  T  -*•  00  .  However,  the  normalized  variables 


(65) 


have  a  joint  density  whose  mass  lias  entirely  along  the  line  a  ■  B  (0<a<l)  .  The 
marginal  distribution  of  either  a  or  B  is: 


(66) 


P{a  <  x}  -  —  sin  \Tx. 

—  7T  v 


0  <  x  <  1  , 
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the  arc-sine  law  so  well-beloved  of  Feller  [8],  Similar  results  hold  for  the 

joint  distribution  of  the  maximum  and  n  r  _  s  ,  ,  or  the  joint  distribution 

pmin  [N, n(T) ]  J 

of  the  maximum  and  minimum  in  (N;T)  (see  [8]  and  [16]  for  corresponding  results 
in  ordinary  fluctuation  theory.) 

10.  GI/G/1 

To  illustrate  an  application  of  renewal-reward  fluctuations,  consider  the 
GI/G/1  queue.  It  is  a  well-known  result  from  queueing  theory  that,  if  customers 

numbered  0,1,2,  ...  arrive  at  epochs  tq  =  Oji^ji^+x^i  ...  and  have  service 

ih. 

times  °o>°;l»02‘  ....  then  the  delay  in  queue  of  the  n  customer ,  6^  ,  is  given 

by  the  recursion  relationship: 


(67) 


6=0 

o 


6  =  [6  ,  +  a  -  t  ] 

n  n-1  n-1  n 


n  =  1,2 ,  ... 


where  [x]  =  max  (x,o)  ,  and  it  is  assumed  there  were  no  customers  present  at  the 

th 

zero  arrival. 

If  we  identify  the  rewards  of  the  fluctuation  process  as: 


(68) 


n  ■  1,2,  ... 


then  we  see  that  p 

n 

If 


n 

l  is  identical  with  6^  as  long  as  6^  stays  positive. 


(69) 


ti  ®(n)  - 


max 


j  -  0,1, 


n 


k0+ 


.©+ 


© 


is  the  last  weak  descending  ladder  index  (including  zero),  then,  in  general. 


6 

n 


p_  -  p  r\ 

77  ®  (n) 


(70) 
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(See  Chapter  6,  [8],  and  Chapter  4,  [15].) 

For  a  GI/G/1  queueing  process  which  begins  with  no  customers  present  at  the 
zero1"*1  arrival,  define: 

u  *  the  number  of  customers  served  in  a  busy  period 
8  =  the  duration  of  a  busy  period 
y  =  the  duration  of  a  busy  cycle 
i  =  y-B  =  the  duration  of  an  idle  period 

it  follows  directly  from  the  above  observations  that: 

\J  -  v 

l  ~  -X 


(71) 

and 

Then 


(72) 


if  the  reward  structure  is  chosen  from  (68) (~  means  "equal  in  distribution").  If 
the  underlying  renewal  process  (2)  is  then  chosen  to  be  identical  with  the  queueing 
arrival  process,  then: 


(73) 


Y 


-  <j> 


© 


6 


9 


and  we  can  use  previous  results,  especially  the  appropriate  form  of  (34)  -  (35). 
If 


(74) 

p^ai  1 

t)  «  s(t)  ; 

/i  -  0,1,2, 

P{t±  £ 

t}  -  A(t)  ; 

U  »  1,2,  . 

then  the  appropriate 

form  of  Q  is 

given  by: 

(75) 

Q(y.t)  = 

t 

/  S (y+u)  dA(u)  , 

0 

{- t  <  y  < 
It  >  0 
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or,  using  lower-case  letters  for  densities: 


d  ,.Q (y , t)  =  s (y+t)  a(t)  dydt 


and 


(76) 


d2  Qn*(y,t)  =  sn  (y+t)  an  (t)  dydt 

y.c 


(n  =  1,2,  . . .) 


over  the  appropriate  range. 

Substituting  in  (34)  -  (35): 


(77) 


EUV^P')  -  1-, 


-l  £  [ e-<e-p>t  dA“‘(t,  f 

n*l  *7  J 


e~W  dSn#(y) 


which  is  implicit  in  a  slightly  more  general  formula  due  to  Kingman  ([12],  p.351). 
From  the  definition  6  *  y-i  ,  an  alternate  form  of  (77)  is  obtained: 


(78) 


_r  V  -SB  +PI 1  , 

Eta  e  e  r  }  *  l-( 


00  t 

-l  £  JJ*  dAn*(t)  ye-(s+P>y  dSn*(y) 


Let 


(79) 


p  -  P{p  >  0}  -  /  [1-Sn* 

n  n  J 


(t)]dAn#(t)  . 


Then  for  the  busy  cycle  to  be  a  recurrent  event,  i.e.,  for  the  busy  period  to 
terminate  with  probability  one,  F®  must  be  an  honest  distribution. 


(80) 


E{?}  -  E { a }  -  E{ t }  <_  0  or  <_  1 


is  the  usual  queueing  criterion;  from  (41),  we  see  that  an  alternate  criterion  is: 
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(®i)  l  ~  U-Pnl  -  00  . 

n=l 

From  (78)  and  (79)  follow  the  well-known  results: 


OO 


Other  results  can  be  expressed  in  terms  of  the  p ,  or  incomplete  integral  forms 
of  (79),  [10]. 

To  obtain  information  about  customer  delay,  let: 

delay  in  queue  of  the  last  customer  whose 

(83)  6(N;T)  ■  6  ^  f ,T>  =  index  is  less  than  or  equal  to  N  ,  and 

1  ’  J  whose  arrival  epoch  is  less  than  or  equal 
to  T  . 

Then,  by  a  direct  application  of  the  reflection  principle  it  follows  from  (70) 
that 

(84)  6(N;T)  -  e+ 

if  the  correspondence  (75)  is  followed.  Direct  results  then  can  be  obtained  from 
(58)  -  (63).  <  and  *  in  (55)  and  (56)  are  then  the  index  and  arrival  epoch  of 

of  the  last  customer  in  (83),  measured  from  the  beginning  of  the  busy  period  in 
progress . 

If  conditions  (80)  are  satisfied  with  a  strict  inequality,  then  a  stationary 
delay  distribute  n  exists  as  N  ■+  00  and  T  -►  00  ,  given  by  the  appropriate  form 


of  (62): 
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Or,  substitution  of  (63)  into  (18)  -  (19)  gives  a  Wiener-Hopf  integral  equation 
which  is  a  generalization  of  a  famous  result  due  to  Lindley  [13]. 


11.  AN  EXTENSION 

It  should  be  clear  from  the  development  that  the  addition  of  the  renewal 
"random  clock"  created  very  little  additional  labor,  since  keeping  track  of  the 
{x^}  was  unrelated  to  the  decomposition  which  was  made  on  the  . 

Similarly,  one  may  generalize  these  results  to  include  an  r-dimensional  vector 
with  components: 


(86) 


(12 
ri  "\W 


(i  “  1,2,  ...) 


with  distribution 

(87) 


Q(t;y)  -  P{Ti_<t;Ciiy} 


/  i  "  1,2,  .. 

(  t  >_  0 

\  -OO  <  y  <  +oo 


In  fact,  the  components  of  the  {x^}  do  not  have  to  be  nonnegative  if  the 
fluctuations  of  the  {t^}  are  not  under  investigation,  i.e.,  if  only  { p ^ }  is 
decomposed  into  dual  ladder  processes. 

Almost  all  of  the  previous  development  carries  through  with  obvious  modifi¬ 
cation;  in  particular,  if  r-dimensional  vectors  of  "epochs,"  +  ,  and  transform 
variables,  8  ,  are  defined  in  the  obvious  way,  then  (34)  becomes: 


(88) 


E{zVa'VpX}  -  1-e 


-i vff  /'•'  /•" *»[*;«"'<"»] 
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with  the  range  of  integration  Y  being  chosen  for  the  appropriate  ladder  process, 
and  T  being  chosen  from  the  appropriate  domain  of  the  {t^}  • 

Exploitation  of  (88)  would  appear  to  hold  much  promise  for  further  results  in 
queueing  theory.  Formal  justification  of  the  manipulations  necessary  could  be  made 
by  appealing  to  powerful  operator  identities  in  Banach  Algebra  developed  by  Wendel 
[22,23]  and  Baxter  [4,5,6], 
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research  models  are  concerned  with  fluctuations  of  various  economic  returns  which 
are  earned  at  random  points  in  time,  and  whose  increments  are  correlated  with 
the  interval  since  the  last  payoff.  Our  generalization  considers  the  fluctuations 
of  a  cumulative  reward  process,  defined  on  an  underlying  renewal  process.  Most 
of  the  classical  results  carry  through,  including  Weiner-Hopf  type  factorization, 
an  Ander sen-Pollaczek-Spitzer  type  identity,  and  certain  Waldian-Pollaczek 
results.  As  applications,  we  find  the  distribution  of  the  maximum  return  over 
a  mixed  index-epoch  horizon,  and  show  how  certain  general  results  for  the 
GI/G/1  queue  follow  directly  from  the  various  three-dimensional  ladder 
distributions. 
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